BBN at TREC Using Hidden Markov Models for Information Retrieval

We present a new method for information retrieval using hidden Markov models HMMs and relate our experience with this system on the TREC ad hoc task We develop a general framework for incorporat ing multiple word generation mechanisms within the same model We then demonstrate that an extremely simple realization of this model substantially outper forms tf idf ranking on both the TREC and TREC ad hoc retrieval tasks We go on to present several algorithmic re nements including a novel method for performing blind feedback in the HMM framework Together these methods form a state of the art re trieval system that ranked among the best on the TREC ad hoc retrieval task and showed extraor dinary performance in development experiments on TREC